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1 . INTRODUCTION 


Consider  a distribution  of  the  continuous  type  with  the 
density  function  f(x;a),  where  o is  a parameter.  The  values 
X.  obtained  in  N independent  drav/ings  from  the  distri- 

■*’hution,  which  will  be  called  the  observations,  are  independent 
random  variables,  all  of  which  have  the  same  density  function 
f(x)a].  Each  particular  sample  Xj.  will  be  represented  by  a 
definite  point  X^j  ■ (x,  , . . , ,Xj.)  in  the  sample  space  ILj  of 
the  Variables  x^,  The  probability  element  of  the  joint 

distribution  of  the  observations  is 

L(x^,  , , • ,x^)  dx^, , ,dXjj.  ■ f (x^^ja) , • (f (Xj^fo)  dz^, . *(13^  (1) 

which  is  equal  to  the  probability  that  the  sample  point  Xjj 
falls  within  the  N-dimensional  interval  dx^...dx^. 

The  function  L is  known  as  the  likelihood  function  of 
the  sample  X^^. 

The  classical  method  of  estimating  the  unknown  parameter 
a by  means  of  the  observations  consists  in  using  a unique 
function  a ■ a(x. , . . .,x^)  of  the  observations  as  an  estimate 
of  o.  The  merit  of  this  estimator  is  appraised  by  its 
variance.  Under  certain  general  conditions,  the  smallest  pos- 
sible value  of  this  variance  is  given  by 

D^n(“^*VN  yOlnf(x}ayftx)^.f(xia)  dx  (2) 

The  ratio  between  this  minimum  value  and  the  actual  vari- 
ance of  o is  called  the  efficiency  of  S. 

The  procedure  of  estimating  an  unknown  parameter  a 
of  a given  distribution  function  by  means  of  the  observations 
will  now  be  considered  from  a somewhat  different  aspect,  viz, 
as  a process  of  deciding  between  several  possible  values  of  a. 

Any  procedure  of  selecting  one  of  a set  of  competing  hypo- 
theses consists  in  choosing  a unique  function  of  the  observa- 
tions, which  will  be  called  the  selector,  and  a set  of  accep- 
tance regions,  one  for  each  of  the  hypotheses.  The  merit  of 
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the  selector  rail  be  appraised  by  means  of  a new  oonoept, 
called  the  reliability  of  the  selector,  as  will  be  demonstra- 
ted in  the  following. 

2.  THE  RELIABILITY  OF  A SELECTOR 

2.1  A Finite  Number  of  Hypotheses 

Any  selector  T(t^, . . . ,tj^)  is  a unique  function  of  k 
test  values.  Bach  particular  value  of  T will  be  repre- 

sented by  a definite  point  in  the  k-dimensional  space  A of 
the  test  values  t^,...,tj^«  If  k^l  the  selector  is  said 
to  bo  univariate,  if  k»2,  bivariate,  etc. 

Consider  the  case  that  wo  have  to  select  one  of  J hypo- 
theses Hl,...,Hj  by  means  of  the  selector  T.  If  Hi  is 
the  true  hypothesis,  then  T has  a particular  density  func- 
tion, which  will  be  denoted  by  f.  (tj^,...,t.  ),  Wo  now  have 
to  choose  j acceptance  regions  Al,...,Aj,  which  are 

parts  of  the  space  A without  common  points.  We  have 

A - EA^  i 0 (3) 

If  the  sign  of  inequality  holds,  then  the  non-empty  region 
(A-EA,  ) will  be  the  acceptance  re^on  of  the  hypothesis  that 
none  oi  the  j hypotheses  is  true. 

The  selection  rule  now  becomes,  that,  if  the  particular 
value  of  T,  the  test  point,  falls  \vithin  Ai,  then  the  hy- 
pothesis Hi  is  accepted  and  all  the  other  hypotheses  rejec- 
ted. 


Let  us  now,  for  a moment,  suppose  that  Hi  is  the  true 
hypothesis,  then  v;o  will  state  this  fact,  that  is,  we  lure 
making  a correct  selection,  each  time  we  obtain  a test  point 
(t.,...,t.)  which  falls  within  the  region  Ai.  The  probabi- 
lity of  this  event,  denoted  by  PHi,  is  given  by 

PHi  - ^f^(t^,...,tj^)dt^...dtj^  (4) 

It  is  obvious  that  this  probability  depends  on  the  choice 
of  Ai.  If  we,  for  instance,  put  A . ■ A,  then  PHi»l,  but 
then  all  other  regions  and  probabilixiea  will  be  equal  to  zero. 
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The  proper  choice  of  the  acceptance  regions  follows  certain 
rules,  which  are  indicated  bolow. 

Let  us  now  suppose  that  we  can  give  preference  to  none 
of  the  competing  hypotheses.  If  we  then  repeat  the  selec- 
tion procedure  many  times,  then  it  is  reasonable  to  assume 
that  each  hypothesis  will  have  the  probability  l/j  of  oc- 
curring, and  the  probability  of  selecting  the  true  hypothe- 
sis will  be  given  by  the  arithmetic  moan  of  all  probabilities 
PHi,  that  is, 


PS  - E PHi/  j 


Also  PS  depends  on  the  choice  of  the  acceptance  re- 
gions, and  it  is  required  to  define  the  particular  set  Ai, 
which  maximizes  PS.  This  problem  will  now  be  examined  for 
several  different  alternatives, 

2.1.1  Univariate  Solectors 


These  selectors  have  one-dimensional  density  functions 
f (t).  For  the  simple  case  of  two  hypotheses  only  (j*2) 

* let  f-,(t)  and  f«(t)  be  represented  by  the  graphs  in 
Pig.l.  ^ 


If  we  now  arbitrarily  choose  as  the  critical  point  to> 
which  separates  the  regions  Al  ■ ( - a3,t  ) and  A2»(t  , oo), 
then  from  (4^  ^ ° 

PHI.  /f^(t)dt  PH2-  PS- (PH1  + PH2)/2 

-00 

Those  formulas  are  valid  for  any  choice  of  acceptance 
regions,  but  a moment’s  reflection  \vill  show  that  PS  will 
be  maximized,  if,  but  only  if,  we  take  as  the  critical  point 
®12’  'the  abscissa  of  the  intersection  between  the 

two  density  functions,  and  thus  defined  by 

^1^°12^  - ^2^®12^ 

It  can  be  concluded  that  these  regions  may  also  be  de- 
fined as  follows: 


I 
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A1  contains  all  roints  of  ths  space  of  T such  that  f.  (t)>f,(t) 

^ - (8) 

A2  contains  all  points  of  the  spaoe  of  T such  that  fgCt)  >fj^(t) 

This  maximum  value  of  PS  will  be  denoted  by  RS  and 
called  the  reliability  of  the  soleotion. 

Equs.  (6)  and  (8)  ore  valid  also  in  the  more  complicated 

oases,  when  there  are  more  than  one  intersection  between  the 

density  functions,  as  illustrated  in  Pig. 2,  whore  A2  is  com- 
posed of  the  two  intervals  A2a  and  A2b. 

The  extension  to  any  finite  number  of  hypotheses  is  im- 
mediate. 

The  reliability  RS  can  bo  put  in  relation  to  the  con- 
cept of  decision  power  DP.  introduced  in  Soi.Rep.Nr.3  of 
Contract  P6IO52-69-C-OO29  [l]  defined  by 

DP  - 1 - Prob(Sl)  - Prob(E2)  . /(f^(t)  - fgCt))  dt  (9) 

♦ 

whore  Prob(Sl)  - the  probability  of  rejecting  the  true  hypo- 
thesis and  rrob(E2)  > the  probability  of  accepting  a false 
hypothesis. 

V/ith  the  notations  in  Pig.3  we  have  for  j-3 

rni  - 1 - 0^  FH2  - l-b^-o^  PH3  - l-bj 

DF(1,2)  - 1- b^- 0^  DP(2,3)  - l-b2- ©2 

After  some  obvious  calculations  ^Te  arrive  at 

RS  - (1+ DP(1,2)  + DP(2,3))/3  (10) 

The  extension  to  any  number  of  hypotheses  is  immediate. 

If  the  arithmetic  medn  of  the  (j-l)  DP-values  is  denoted 
by  E(DP),  then 

RS  - (!•►  (j- l)E(nP))/j  (11) 
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from  which  it  can  be  concluded  that  with  increasing  J 
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RS  — E(DP) 

2.1.2  ;.\altivariatc>  Seloctors 


In  accordance  v/ith  the  preceding  argumentation  it 
follows  that  for  multivariate  seleotors  we  have 

FHi  - y'f^(t^,...,tj^)  dt^  ...  dtj^  and  RS-EPHi/j  (13) 

where  Ai  contains  all  points  of  the  k-'dimensional  space 
of  T satisfying  the  inequality 

f i ( t^  j . . • , t j^)  > ^h^ ^1  * * * * * (^  / ^)  (id) 

2.2  An  Infinito  Number  of  Hypotheses 

The  preceding  formulas  will  now  be  extended  to  the 
case  of  an  infinito  number  of  hypotheses,  a problem  which 
arises,  when  wo  have  to  select  the  true  value  of  an  un- 
known parameter  «,  v/hich  can  take  any  value  belonging  to 
a non-degenerate  interval. 

In  this  particular  case,  the  coordinates  t^,...,tj^ 

of  the  selector  are  unique  functions  t ■ g^(Xj , . . . ,3^) 

of  the  observations  i. . 

1 

The  study  will  be  started  with  the  most  simple  se- 
lector T«X  , that  is,  taking  the  sample  point  as  the 
tost  point  ‘ withv'ut  any  transformations,  which  implies 
t^  • and  k ■ '« . 

Let  Ki  be  the  hypothesis  that  a is  the  true  value 
of  the  unlaiown  parameter  a>  The  density  function  will 

then  be  given  by 

fi(il,...,x^)  - f(x^jo^)...f(Xj^|o^)  (15) 

2.2.1  Univ:iriate  Solectors 

If  only  one  observation  is  available,  then  the  density 
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functions  becomes  f(x;a. ).  Now  let  the  density  functions 
f(x;a-da),  f(x{a)  and  ^ f(x;a  + da)  bo  reprosontod  by  the 
graphs  in  Pig. 4.  Maximum  reliability  is  attained  only  if 
wo  choose  the  acceptance  region  Aa  ■ (0^,02) i defined  by 

f (o^ja  - da)  - and  f (o2>a)  ■ f (o2{o  + do)  (16) 

For  small  da  we  mai'  put 

f(o^>a-da)  - f(o^ja)  - f*(c^}a)  do 

f(o2}a+da)  - f(c^(a)  + f^(o^fa)da 

where 

f^(x}o)  - 3f(i;a)/3o  (17) 

from  which  it  follows  that 

f’(o-i;o)  - " ^■*(050)  - 0 (18) 

a 1 a c a 

that  is,  v/hen  da  — 0,  then  and  O2  tend  to  the 

same  value  0,  which  is  the  abscissa  of  a point  common 
to  f(x;a)  and  the  envelop  of  the  family  of  the  density 
functions,  as  indicated  in  Pig. 4.  This  result  implies 
that  the  acceptance  region  (o^jCj)  degenerates  into  the 
point  0 and  the  selection  rule  becomes  that,  if  we 

have  a single  observation  x^,  then  we  select  as  the 

true  value  of  a the  particular  value  a,  which  is  given  by 

(Xjfo)/  9o  - 0 (19) 

Observing  that  f(x}a)  is  the  likelihood  function  of 
a sample  of  size  N-1,  it  follows  that  o is  iden- 

tical with  ■*  the  maximum  likelihood  estimate,  which  thus  has 
been  proved  to  have  maximum  reliability  in  this  particular  case. 

The  selection  laile  (19)  niay  also  be  put  in  the  form 

3ln  f(i^|o)/3a  ■ 0 (20) 
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Let  US  now  suppose  that  a can  take  a very  large  num- 
ber of  discrete,  equidistant  (distance  ■ da)  values  a.. 

As  demonstrated  in  earlier  publications,  CP  may  then  be 
replaced  by  the  estimation  power  EP.  The  reliability  of 
will  then  be  given  by 

RS  - E(EP(a)  ) da  (21) 

Prom  (9)  it  may  bo  derived  that 

EP(a)  - /f»(xjo)dx  (22) 

if  * 

where  the  integration  includes  all  points  x with  positive 

values  of  f*  , as  indicated  by  the  -t-sign. 
a ’ 

Since  EP(a)  is  a function  of  a,  typical  for  each 
value  of  a,  the  mean  E(EP)  may  be  replaced  by  an  integral. 

It  will,  however • be  preferable,  as  being  more  informative, 
to  use  the  EP(a) — funotion  itself  as  a measure  of  the  reliabi- 
lity of  a selector,  as  vrill  be  Illustrated  in  the  sequel. 

The  question  now  arises  whether  it  will  be  possible  to 
increase  the  reliability  by  introducing  a transformation 

y - g(x)  (23) 


of  the  observations  x. 

Two  necosaary  conditions  will  bo  imposed  upon  the 
funotion  g(x); 

1)  there  must  be  a uniquely  defined  y correlated  with  each  x, 

2)  no  two  of  the  transformed  acooptanoo  regions  may  have  com- 
mon points. 

Those  two  conditions  ore  satisfied,  if  the  function  g(x) 
defines  a biunique  mapping  of  the  domain  of  y onto  that  of 
X,  v/hloh  holds  if  g(x)  is  monotone,  i.o.  steadily  increasing 
or  steadily  decreasing  as  x increases,  as  illustrated  in 
Pig. 4.  If  a is  the  true  value,  then  the  probability  of  se- 
lecting it  is  equal  to  the  area  of  the  shaded  region,  that  is, 
to  the  probability  that  a value  x drawn  from  a distribution 
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with  the  donoity  function  fCxio)  fella  within  the  interval 
(o,  ,Op).  V/ith  oaoh  auoh  value  there  ia  alv/aya  correlated  e 
valuo*^  y which  fulls  within  the  tranaformod  aoooptunoe  region 
(s(Oi)tC(oo))f  ooncludod  that  RS  la  invariant 

under  any  nooeptablo  transformation.  Consequently  no  improve- 
ment of  the  reliability  is  possible  by  means  of  transfor- 
mations of  the  observations. 

2.2.2  Pi variate  and  Multivariate  Solootors 


liot  us  now  suppose  that  two  observations  are  available. 
Taking  Xp  as  the  aoloctor,  the  density  function  correspon- 
ding to  the  hypothesis  Ha.  that  a.  is  the  true  value  of 
a,  will  bo  given  by 

fj(x^(X2ja)  “ f(x^|a^)  . f(x2|aj^) 

Comparing  throe  adjacent  density  functions,  as  in  the 
preceding,  corresponding  to  o-da,  a and  a -f  da,  it  fol- 
lows that,  when  da  — 0,  the  aoooptanoe  region  Aa  dega- 

neratos  into  a curve  in  the  Xj,X2-plRne. 

The  selection  rule  then  becomes  that,  if  we  have  two 
observations  x^  and  x^,  then  we  will  seleot  us  the  true 
value  of  a the  particular  value  a,  which  satis- 

fies the  condition 

^[in  f(xjta)  + In  fCx^f  a)]/  Ba  ■ 0 

The  extension  to  any  number  of  observations  is  Immediate, 
being 

^ [in  f(Xj^ja')  + ...  f(Xjj|o)]/3o  ■ 0 

The  selected  value  a is  identical  \vith  the  maximum 
likelihood  estimate. 

The  estimation  power  KP(a)  will  be  given  by 

EP(a)  - ^^[  3(f(x^}^)...f(xjj|S)/3a]dx^...dXjj 

where  the  integration  ia  taken  over  all  points  with  a positive 


value  of  the  partial  dorivative,  as  indicated  by  the  +8ign. 


It  can  be  proved  that  EP(«)  is  invariant  under  any  ac- 
ceptable transformation  y^-e(xj)»  of  the  observations,  which 
implies  that  no  improvement  of  the  reliability  oan  be  made 
in  this  way. 

Let  us  now  examine  the  effect  of  rearranging  the  elements 
of  the  sample  in  ascending  order  of  magnitude,  de- 
noting  them  by  and  calling  them  the 

order  statistics  in  the  ' ■'sample,  ' 

The  probability  element  of  the  joint  distribution  of 
arbitrary  set  of  order  statistics  is  given  by  Sarhan  & 

Greenberg  [2],  In  particular  we  have,  if  all  order  statistioe 
are  taken,  the  density  function 

N:[f(x^^^...f(r(jjj]  (28) 

which  differs  from  the  unarranged  sample  only  by  the  factor 
NJ.  The  selection  rule,  indicated  by  equ.(26),  will  thus 
result  in  the  same  selected  value  a* 

The  introduction  of  the  order  statistics  has,  hewever, 
the  advantage  of  making  it  possible  to  censor  or  truncate 
the  sample.  V/e  may  even  use  a single  order  statistic  of  the 
sample.  The  estimation  power  EP(tf)  depends  very  much  on  the 
order  number,  thus  indicating  where  the  information  is  located 
within  the  sample. 

The  preceding  general  formulas  will  now  be  applied  to  the 
V/eibull  distribution  and  further  developed, 

3.  APPLICATION  TO  THE  WEIBULL  DISTRIBUTION 
3,1  One  Unknown  Parameter 

A single  unknown  parameter  may  be  determined  by  means  of 
a single  observation,  as  will  now  be  demonstrated.  The  den- 
sity function  of  the  selector  is  given  by 

f(x,m,0,^)  - (m/p)  a®”  ^ e"®  (2o'. 
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j 


r 
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where 


z 


(30) 


and 


(x-  ^)/3 


m m l/a  ■ the  shape  parameter 

(1  ■ the  scale  and  the  location  parameter 

Prom  (29)  we  have 

In  f (x)  ■ In  ro  - In  P + (m-l)  In  a - 

3«1.1  The  Parameter  tt » l/m  Unknovm,  P and  m Known 

Prom  (31)  it  follows  that 

‘Jlnf(xy3a  - -m^  Qln  f (x)/ 3m--  m(  1 + In  a® - a™ In  a®) 
Introducing 

m o 

u - a a - u 

a * 1 

dx-a.p*u  du 

a.P.f(x)  - ) f(x)  dx 


wo  have 


-0  "^In  f(x)/3o  - t(u)  - 1 + Inu  - u.lnu 

Some  values  of  the  function  t(u)  are  Hated  in  Table  1. 
17e  have  t(u^)  - t(u^)  - 0 for 

. u^  - 0.25924  # - 2.23893 

Hence,  if  a single  observation  x^  is  available,  then  it 
follows  from  equ.(20)  that  the  selected  value  m is 
given  by  ^ 

<»  m. 

X®'  - 0.25924  or  x^  - 2.23893 


(31) 


(32) 


(33) 


(34) 
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Bca 


or 


1.70562  logx^  or  l/mg  ■ Og  ■ 2.85683  logx^ 

Siroo  always  a^O^  the  value  0^  is  usedf  when  x^4lf 
and  Ogf  when  x^  >1. 

The  estimation  power  of  o is  from  equ,(22)  given  by 

EP(o)  ■ f(x)/3a)  dx  - yl[  3ln  f(x)/d  o)f(x)  dx 

f -*■ 

Thus 

o.EP(a)  - y(l  + Inu  - u.lnu)e““  du 
Observing  that 

d(u.lnu.e'’'^)  • (l  + Inu  - u.lnu)e"“du 
it  follows  that  11^ 

a.EP(^)  - / (u  .Inu.e'^  ■ 0.46237 
3.1.2  The  Paramotor  B Unknov/n.  g and  y Known 
Prom  (31)  we  have  after  some  easy  oaloulationo 
a. 3 ^ln(f(x)/^  3 - u-  1 
The  selection  rule  then  becomes 

u - ((Xj-  ji)/^“-l 


or 


or 


3-*i-^ 

A 

The  estimation  power  ^EP(p)  is  given  by 

a.3.EP(3)  - /(u-  1)  du  - 0.36788 

/ 

EP(3)  - 0.36788/a. 3 


(35) 


(36) 


(37) 


(38) 


(39) 


(40) 
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The  Parameter 


From  (31)  we  have  after  some  easy  calculations 
a.p  ^In  f(x)/^u  - (u-(l-o))/u® 

A 

The  selected  value  ^ will  he  given  by 

« - ((*!“  Ji)/0)“-  l-« 

or 

^ - Xj-  p(l-  «)“ 

The  estimation  power ^ EP(  ji)  is  given  by 

a.(3.EP(u)-  y (1  - o - u)  u*.  e"'*  du 
*& 

■ 0 for  a > 1 
- 0.36788  for  o ■ 0 

3.2  All  Parameters  Unknown 

Maximum  reliability  is  attained , if  we  choose 
as  the  selector.  Its  density  function  L is  given 
by  equ  (l).  Introducing  equ.(29)  we  have 

L - TX  m(x  - ^ 

A A A 

The  selected  values  m,  ^ and  p are  obtained  by  equating 
to  zero  the  partial  derivatives,  that  is,  by  solving  the 
system  of  equations 

•CL/^  m»^;>L/'^  u ■ 0 and  ^L/ ^ P - 0 

A 

Prom  the  last  aquation  the  value  p will  be  given  by 


p"  . E (x^  - ^r/N 


Introducing  (46)  into  (44)  and  neglecting  factors  depending 
on  K only,  we  arrive  at 
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L ■ TC  m(x.  - |i)®  (47) 

{•I  ^ * 

In  the  particular  case  Mrtien  ^ ■ 0 *•  have 

L - TCm.x“'V2x  “ (48) 

^ 

Instead  of  solving  the  system  of  equs.(43)f  i'k  teen  found 
convenient  to  compute  L in  equ.(47)  for  e properly  ohoeen 
set  of  m and  ^u-  values  and  to  seleot  the  particular  pair 
m,  p which  maximizes  L. 

To  this  purpose  the  computer  program  6/73  has  been  writ- 
ten and  applied  to  a large  number  of  samples  of  fatigue  test 
data  collected  at  the  Boeing  Company,  as  will  be  reported 
elsewhere. 


The  computing  time  for  a complete  evaluation  of  euoh 
samples  of  size  V ■ 10  Is  only  about  one  second. 


I 


n 


4.  REFERENCES 


1.  W*lbull,  W.:  "Outlln*  of  a Theoz7  of  Powerful  Selection  of  Distribution 
Functions",  AFML-TR-71-52,  March  1971. 

2.  Sarhan,  A.E.  & Greenberg,  B.G.:  "Contributions  to  Order  Statistics", 

J.  Wiley  & Sons,  Inc.,  New  York,  London,  1962. 


-14- 


■BBSr 


TABLE  1.  THE  FUNCTION  t(u)  ■ 1 Inu  - u.lnu  (u  ■ t*°) 


u t(u) 

u 

t(u) 

u 

t(u) 

u 

t(u) 

.00  “ 00 

0.0 

- 00 

1.0 

1.00000 

2.0 

0.30685 

.01  -3.55912 

0.1 

-1.07233 

1.1 

0.99047 

2.1 

0.18387 

.02  -2.83378 

0.2 

-0.28754 

1.2 

0.96354 

2.2 

0.05385 

.03  -2.40136 

0.3 

0.15722 

1.3 

0.92129 

2.3 

-0.08276 

.04  -2.09012 

0.4 

0.45023 

1.4 

0.86541 

2.4 

-0.18483 

.05  -1.84594 

0.5 

0.65253 

1.5 

0.79727 

2.5 

-0.37912 

.06  -1.64461 

0.6 

0.77567 

1.6 

0.71800 

2.6 

-0.52883 

.07  -1.47311 

0.7 

0.89300 

1.7 

0.62856 

2.7 

-0.68854 

.08  -1.32363 

0.8 

0.95537 

1.8 

0.52977 

2.8 

-0.85838 

.09  -1.19125 

0.9 

0.98946 

1.9 

0.42233 

2.9 

-0.94404 

.10  -1.07233 

1.0 

1.00000 

2.0 

O.3O685 

3.0 

-1.19722 

t(u)«0  for  u^t»  0.25924  and  2,23893 
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